Network controller, a multi-fabric shelf and, a method of processing traffic in a transport network

ABSTRACT

A network element of a transport network has three fabrics housed within a single shelf of a telco rack, namely a packet fabric, an electrical fabric and an optical fabric. A stream of traffic including a plurality of lambdas is received at a trunk interface of such a shelf. The optical fabric in the shelf performs optical switching on the stream to replace a first lambda in the stream with a second lambda. The first lambda is converted within the shelf into an electrical signal. Also within the shelf, first frames are recovered from the electrical signal. The packet fabric in the shelf is used to perform packet switching on the first frames to generate a flow of second frames. The flow of second frames is transmitted at a client interface of the shelf.

CROSS REFERENCE TO PARENT APPLICATION

This application is a continuation application of U.S. application Ser.No. 13/405,330 filed on Feb. 26, 2012 by Stephen J. West and ScottPradels, and entitled “Method of Processing Traffic In A TransportNetwork” that in turn is a divisional application of U.S. applicationSer. No. 12/148,281 filed on Apr. 16, 2008 by Stephen J. West and ScottPradels, issued as U.S. Pat. No. 8,155,520 and entitled “MULTI-FABRICSHELF FOR A TRANSPORT NETWORK”. U.S. application Ser. No. 13/405,330 andU.S. Pat. No. 8,155,520 are both incorporated by reference herein intheir entireties.

RE-VISIT NOTICE

Applicant hereby rescinds any disclaimer of claim scope in theabove-identified parent application (namely U.S. application Ser. No.13/405,330) or grandparent application (namely U.S. application Ser. No.12/148,281) or the corresponding prosecution histories thereof andadvises the US Patent and Trademark Office (USPTO) that the claims inthe current application may be broader than any claim in the parentapplication or the grandparent application. Applicant notifies the USPTOof a need to re-visit the disclaimer of claim scope in the parentapplication and grandparent application, and to further re-visit allprior art cited in the parent application and grandparent application,including but not limited to cited references over which any disclaimerof claim scope was made in the parent application or grandparentapplication or the corresponding prosecution histories thereof. SeeHakim v. Cannon Avent Group, PLC, 479 F.3d 1313 (Fed. Cir. 2007).Moreover, any disclaimer made in the current application should not beread into or against the parent application.

BACKGROUND

A transport network is typically composed of nodes at various locationsthat are interconnected by optical fibers, to transport traffic overlong distances (such as distances between cities). The traffic includesdiscrete units of data e.g. conforming to Internet Protocol (IP), aswell as time-sensitive streams of data as in voice circuits intraditional telephony. Each node in the transport network typicallyprovides one or more functions for creating signals which aretransmitted on the optical fibers.

FIG. 1 illustrates a prior art network which implemented using routersto process IP packets, using multiservice provisioning platforms (MSSPs)to generate optical signals, and using optical add drop multiplexers totransport the optical signals across the transport network. In thearchitecture illustrated in FIG. 1, a flow of packets from node A tonode Z is typically processed at each intermediate node, I, J and K. Anoptical signal generated by node A may contain not only packets for nodeZ but also packets for intermediate nodes I, J and K. In addition topacket traffic, an optical signal from node A to node I may also containtime-division-multiplexed (TDM) traffic, such as SONET.

The inventors of the current patent application note several drawbacksin using a combination of routers, MSSPs that lack an optical backplane,and OADMs as shown in FIG. 1. Specifically, the current inventorsbelieve that router-based transport is expensive, consumes preciousresources, and compromises network performance due to higher latency.Moreover, the current inventors believe that MSSPs being based onSONET/SDH circuit-based hierarchy are obsolete because packet trafficover transport networks is increasing significantly relative totraditional telephony voice traffic. For example, increase in packettraffic is caused by residential users' demand for packet-basedapplications, such as transport of video over the Internet. As anotherexample, increase in packet traffic is caused by business usersexpanding enterprise-wide Ethernet based networks to reach across theInternet. Accordingly, the inventors of the current patent applicationbelieve there is a need for improvement in network elements of atransport network.

SUMMARY

In accordance with the invention, a network element of a transportnetwork has three fabrics housed within a single shelf of a telco rack,namely a packet fabric, an electrical fabric and an optical fabric. Astream of traffic which includes a plurality of lambdas is received at atrunk interface of such a shelf. The optical fabric in the shelfperforms optical switching on the stream to replace a first lambda inthe stream with a second lambda. The first lambda is converted, withinthe shelf, into an electrical signal. Also within the shelf, firstframes are recovered from the electrical signal. The packet fabric inthe shelf is used to perform packet switching on the first frames togenerate a flow of second frames. The flow of second frames istransmitted at a client interface of the shelf. The shelf of someembodiments includes inter-fabric circuitry, to bridge between thefabrics. The inter-fabric circuitries switchably transmit packets acrossfabrics in intermediate nodes of the transport network.

In several embodiments of the invention, circuitry in such a shelf addsnew headers to packets for transmission across the transport network ina connection oriented manner. An example of the new header (calledbackbone header) is a media access control (MAC) header which is newlyadded to a frame that already has a MAC header, to perform MAC-in-MACencapsulation for use within the transport network. At the destinationnode, the added header is removed before supplying packets to an enduser or other client.

In some embodiments of the invention, circuitry in the shelf aggregatesthe information in multiple electrical signals from an electrical fabricin the shelf, and thereafter frames the aggregated signal with errorcorrecting code(s), followed by its conversion into a lambda. The errorcorrection scheme is predetermined, to add redundant informationsufficient to overcome attenuation due to a lambda traversing OADM(s)without regeneration.

Presence of three different types of fabrics in a single shelf inaccordance with the invention enables optimization across fabrics at anunprecedented level, e.g. optimizing usage of a lambda at the level ofpackets. Hence, the optical fabric and its external interfaces are usedin certain embodiments as an all-optical cross-connect to perform lambdaswitching within the shelf at an intermediate node (also called OOOswitching). At another intermediate node if the signal to noise ratio ofa lambda has degraded, the optical fabric and the electrical fabric areused together in the single shelf of that intermediate node, to performOEO switching and/or regeneration. By use of OOO and/or OEO switching,packet traffic can be sent from a source node to a destination nodewithout recovery of individual packets at intermediate nodes.Elimination of individual packet retrieval and processing atintermediate nodes reduces cost, resource usage, and latency, whencompared to use of routers, MSSPs and OADMs. Optimization in packetprocessing, by its elimination from one or more intermediate nodes, isbelieved to be nowhere disclosed or rendered obvious by any prior artknown to the inventor(s) of the current patent application.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates, in a block diagram, an architecture of a transportnetwork based on routers, MSSPs and OADMs in the prior art.

FIG. 2A illustrates, in a block diagram, an architecture of a transportnetwork in accordance with the invention that transfers packets directlyon a lambda from a source node A to a destination node Z without packetprocessing at intermediate nodes.

FIG. 2B illustrates, in a block diagram, the architecture of FIG. 2Awherein a lambda carrying packets from a source node A is terminated andregenerated at intermediate nodes I and K before reaching destinationnode Z, again without packet processing at the intermediate nodes.

FIG. 3A illustrates, in a block diagram, a single shelf 100 that usesthree fabrics to transfer packets directly to another single shelf 110,to implement the architecture shown in FIG. 2A.

FIGS. 3B and 3C each illustrate physical layout inside a single shelf inalternative embodiments in accordance with the invention, whereinfabrics are located in a rear region and inter-fabric circuitry andexternal interfaces are located in a front region, with a midplanebetween the two regions.

FIG. 3D illustrates the shelf of FIGS. 3B and 3C mounted in a telcorack.

FIG. 3E illustrates a front view of a shelf in one illustrativeembodiment in accordance with the invention.

FIG. 4A illustrates, in a block diagram, a wavelength selective switchand optical cross-connect implemented within a shelf of in someembodiments of the invention.

FIG. 4B illustrates, in a block diagram, a packet fabric implementedwithin a shelf of in some embodiments of the invention.

FIG. 4C illustrates, in a block diagram, an electrical fabricimplemented as a crossbar within a shelf of in some embodiments of theinvention.

FIG. 4D illustrates, in a block diagram, an electrical fabricimplemented as a TDM fabric within a shelf of in some embodiments of theinvention.

FIG. 4E illustrates, in a block diagram, a hybrid electrical and packetfabric implemented using a crossbar within a shelf of in someembodiments of the invention.

FIG. 5A illustrates an optical interface module (called “LXC”) thatinterfaces optical signals from a trunk fiber (e.g. carrying fortylambdas in OCh format of OTN that have been wave division multiplexed)to the optical fabric in a shelf of some embodiments of the invention.

FIG. 5B illustrates another optical line card (called “LMX”) thatreceives optical or electrical tributary signals (such as OC48 or STM 16or OTU1) from outside the shelf and supplies them to the optical fabricin some embodiments of the invention.

FIG. 5C illustrates two cards (called “PSW” and “PMX”) that are coupledto packet and electrical fabrics in a shelf of some embodiments of theinvention.

FIGS. 5D and 5E illustrate an electrical crossbar and an opticalcrossbar respectively that are used within a single shelf in combinationwith a packet fabric in accordance with the invention.

FIG. 5F illustrates a front view of an optical fabric module 215 used insome embodiments.

FIG. 5G illustrates detail of a single optical connector which islabeled 5G in FIG. 5F, with two vertical rows of fibers, including aleft row for input signals labeled IN3-1, 1N3-12 and a right row foroutput signals labeled OUT3-1 and OUT3-12.

FIG. 5H illustrates a cross-sectional view along the direction Z-Z inFIG. 5I.

FIG. 5I (including FIGS. 5I-1 and 5I-2) illustrates a top view of theoptical fabric module 215, with the lid removed, wherein optical fibersare illustratively shown connecting four connectors to a laminate whichholds the fibers in an optical flexplane; in this view, dimensions areshown in millimeters.

FIGS. 6A and 6B illustrate a side view and a rear view respectively, ofthe shelf of FIG. 3E.

FIGS. 6C and 6D illustrate a front view and a perspective of a frontregion of the shelf of FIGS. 6A-6B, the front region being used to holdexternal interfaces and inter-fabric circuitry.

FIGS. 6E and 6F illustrate a rear view and a perspective of a rearregion of the shelf of FIGS. 6A-6B, the rear region being used to holdfabrics and copper interfaces.

FIG. 6G is same as FIG. 6C with the addition of a section line A-A alongwhich is shown a cross-sectional view in FIG. 6H.

FIG. 6H is a cross-sectional view along section line A-A of FIG. 6G.

FIG. 6I is an exploded view of the shelf of FIGS. 6A-6G, wherein top andside portions of the shelf are removed to improve clarity.

DETAILED DESCRIPTION

In accordance with the invention, a network element of a transportnetwork has a single shelf that holds as many fabrics (e.g. N fabrics)as the number of switching modes (e.g. N switching modes) used tomultiplex traffic on a link between nodes in the transport network. Thenetwork element has one or more external interfaces to receive/transmittraffic to/from the shelf. The external interfaces can be of at leasttwo types: trunk interfaces and service interfaces. The shelf alsocontains inter-fabric circuitry that bridges traffic between fabrics.External interfaces and inter-fabric circuitry of the network elementexchange traffic between each other through the multiple fabrics in theshelf, based on provisioning by an external network controller.

One illustrative embodiment of the invention uses three multiplexingtechnologies on links between nodes, namely (a) multiplexing of discreteunits of data (called packets) to form an electrical signal to betransmitted across the transport network (packet multiplexing intoflows), (b) multiplexing of packet-carrying electrical signals withTDM-carrying electrical signals to form an aggregate (e.g. an ODU2frame) to be transmitted on a lambda (sub-lambda electricalmultiplexing), and (c) dense wave division multiplexing (DWDM) oflambdas within an optical fiber. In this illustrative embodiment, asingle shelf 100 (FIG. 3A) houses three types of fabrics: (a) a packetfabric 102 (such as a switch fabric) (b) an electrical fabric 104 (suchas an analog electrical crossbar) and (c) an optical fabric 106 (oroptical crossbar, such as a passive optical mesh with wavelengthselective switch endpoints).

In the illustrative embodiment, optical fabric 106 in shelf 100 isimplemented as a crossbar in the optical domain that switches opticalsignals. Such an optical fabric may be used to perform opticalswitching, on a stream of traffic (comprising a plurality of lambdas)received from a trunk interface, to replace a first lambda in theplurality of lambdas with a second lambda. An example of such an opticalcrossbar is a passive optical mesh that optically connects all opticalinterfaces and electrical-optical inter-fabric circuitry to one anotheras illustrated in FIG. 5E, by use of optical fibers that are laminated.In one illustrative embodiment of the type illustrated in FIG. 5E, eachof the 24 end-points includes a fiber that provides an opticalconnection to each of the other end-points. Referring to FIG. 3A,optical fabric 106 optically connects electrical-optical inter-fabriccircuitry 105A-105M, external interfaces 107A-107Z (connected to fibersexternal to shelf 100), and optionally packet-optical inter-fabriccircuitry 109A-109R (if implemented). The passive optical mesh can beused in an optical fabric module 215 in some embodiments, as illustratedinn FIGS. 5F-5I. The optical fibers of one illustrative embodiment areSMF-28, single mode, with maximum insertion loss @1550 nm per connectorper channel being 1 dB, and maximum return loss @1550 nm per connectorper channel being −50 dB. FIG. 5G illustrates in detail, the front viewof slot 3 in shelf 100, which is typical for the remaining slots. Notethat slots 10, 12, 14 and 16 in FIG. 5F are not used in the illustrativeembodiment. Also note that in the illustrative embodiment, the fibersare laminated to form a flexplane which is installed in an enclosurewith fiber facing down. Also mounted in the enclosure are connectors ofthe type shown in FIG. 5G.

Each of external optical interfaces 107A-107Z includes a tunable laserand tunable optical filter or an optical splitter, and a wavelengthselective switch (WSS), configured to implement dense wave divisionmultiplexing (DWDM). Optical fabric 106 and interfaces 107A-107Z areused in some embodiments as an all-optical cross-connect to performlambda switching within shelf 100, to switch any wavelength from any ofinterfaces 107A-107Z to any of the other interfaces 107A-107Z, toimplement OOO switching. Moreover, electrical-optical inter-fabriccircuitry 105A-105M (e.g. LMX) are used in some embodiments incombination with optical fabric 106 and its interfaces 107A-107Z (e.g.LXC) to terminate a wavelength and recover electrical signal(s),followed by retransmitting the electrical signal(s) on the same ordifferent wavelength, to implement OEO switching.

In the certain embodiments, electrical fabric 104 in single shelf 100 isalso implemented as a crossbar (FIG. 5D), although in the electricaldomain. The electrical fabric 104 switches electrical signals (a) thatare recovered from lambdas and/or (b) that will be converted intolambdas. The electrical fabric 104 and electrical-optical inter-fabriccircuitry 105A-105M implement switching in a layer below lambdaswitching in the optical domain. Hence electrical fabric 104 and itsinterfaces in circuitry 105A-105M, 103A-103Y, 108A-108P are togetherreferred to herein as a sub-lambda electrical switch. The electricalfabric 104 supports switching of any format electrical signals,independent of content therein. Specifically, the electrical fabric 104does not distinguish between an electrical signal that carries TDM dataand another electrical signal that carries packet(s) and/or fragments ofpackets.

In the certain embodiments, a packet fabric 102 (FIG. 3A) in shelf 100includes a switch fabric. External interfaces 101A-101X to packet fabric102 include traffic managers that classify packets to be switched bypacket fabric 102, and form flows that are directed to specificdestinations within the shelf. To improve efficiency in use of theswitch fabric, packets are fragmented by the traffic managers andthereafter transferred across the switch fabric to traffic managers inother packet fabric interfaces 101A-101X or to inter-fabric circuitry103A-103Y that bridges the packet fabric to the electrical fabric.Traffic managers in the packet-electrical inter-fabric circuitry103A-103Y reassemble packet fragments into whole packets, and beforetransferring each packet through the transport network.

Operation of shelf 100 is now described in the example shown in FIG. 3A.Specifically, shelf 100 has a service interface 107A coupled externallyvia a router to a content provider that supplies an optical signalcontaining for example, video content of interest to various end users.Packets within such an optical signal at service interface 107A need tobe retrieved prior to being routed to the appropriate end user. Hencethe optical signal is first terminated in a selected one ofoptical-electrical inter-fabric circuitry 105A . . . 105M. The specificcircuitry which is used for termination of this optical signal isprovisioned by a network controller 120 of the telecommunicationscarrier that manages shelf 100. Next, the electrical signal which hasbeen retrieved (from the optical signal) is itself switched viaelectrical fabric 104 to one of the packet-electrical inter-fabriccircuitry 103A-103Y. The identity of which circuitry is to be used fortermination of an electrical signal to recover packets is provisionableby network controller 120. The packets are thereafter switched by packetfabric 102 to a suitable one of the packet-electrical inter-fabriccircuitries 103A-103Y.

The packet-electrical inter-fabric circuitry creates an electricalsignal and switches the electrical signal through the electrical fabric104 to an appropriate one of the optical-electrical inter-fabriccircuitries 105A-105M. The electrical-optical inter-fabric circuitrycreates an optical signal that is switched by the optical fabric to anappropriate one of the external optical interfaces 107A-107Z, followedby transmission via the rest of the transport network, e.g. to adestination shelf 110. Shelf 110 to which an end-user is coupled issimilar or identical in architecture to the above-described single shelf100, and operates similarly.

In certain embodiments, inter-fabric circuitries add information priorto transmission of packets and/or electrical signals through thetransport network. Specifically, prior to conversion into a lambda,electrical-optical inter-fabric circuitry 105A-105M frames an electricalsignal by adding bits/bytes. The framing which is added to an electricalsignal includes extra redundant bytes to be used by a destination shelf(where the lambda is terminated) to detect and correct errors in thereceived signal, so that the original signal (prior to framing) isrecovered.

In one illustrative embodiment, the extra redundant bytes are used by adestination shelf to implement a forward error correction (FEC)technique to increase the signal to noise ratio (relative to theuncorrected signal) to a level sufficient to overcome Optical Signal toNoise Ratio (OSNR) degradation arising from the lambda traversing one ormore OADMs without regeneration. While any forward error correctiontechnique can be used in various embodiments of the invention, oneembodiment of electrical-optical inter-fabric circuitry 105A-105M addsindustry-standard framing, as described in ITU-T G.709 entitled,“Interface for the optical transport network (OTN)”.

Hence one embodiment adds framing bits using Reed-Solomon codeRS(255,239) with 8-bit symbols, wherein 239 is the number of symbols,and 255 is the total number of symbols per codeword, and each codewordconsists of data and parity (also known as check symbols) that areadded. In this embodiment, a G.709 framer maps four OC48 signals intofour ODU1 frames, and thereafter maps the four ODU1 frames into a singleODU2 frame, that is then supplied as an OTU2 signal to an opticaltransceiver for creation of a lambda.

Moreover, packet-electrical inter-fabric circuitry 103A-103Y of someembodiments add a new header that is local to the transport network, foruse in transmission of a flow of packets in a connection oriented mannerthrough intermediate nodes in the transport network. In one illustrativeembodiment, the original packets are Ethernet frames that contain mediaaccess layer (MAC) headers, and new headers (also called backbone MACheaders) have the same structure as Ethernet's MAC header. Specifically,traffic managers of packet-electrical inter-fabric circuitry 103A-103Yof some embodiments implement MAC-in-MAC encapsulation of each packet,after classification etc.

A flow of encapsulated packets through the transport network follows apath identified in each intermediate node based on one or more field(s)in the backbone MAC header until the destination is reached. Oneembodiment uses the backbone MAC address and the backbone VLAN id andperforms a 60-bit lookup to identify the path. At the destination, thebackbone MAC header is stripped off from the packets which arethereafter forwarded to an end user who's outside the transport network,in the normal manner. The path for each flow through the transportnetwork is provisioned at each intermediate node through which anencapsulated packet passes. The provisioning is done by a networkcontroller of a telecommunications carrier that operates the transportnetwork. Provisioning of paths for MAC-in-MAC encapsulated packetseliminates the need for Ethernet discovery mechanisms, which are notused to route one packet at a time through the transport network.

Depending on the embodiment, packet-electrical inter-fabric circuitry103A-103Y may be implemented in conformance with any known protocol,such as Provider Backbone Bridges (PBB) and/or Provider BackboneTransport (PBT), and/or Provider Backbone Bridging-Traffic Engineering(PBB-TE). PBB-TE is currently being standardized by the IEEE 802.1 Qaytask force, e.g. as described in Draft 2.0 specification that has beenreleased recently. PBB-TE operation which is used in one embodiment ofthe invention differs from PBB as follows. In PBB, the backbone MACheader's VLAN identifier B-VID identifies a packet flooding domain whichinterconnects different PB networks. In the PBB-TE, the B-VID incombination with the backbone MAC header's destination address B-DAaddress, identifies a specific path through the transport network. Thespecific path which is identified in PBB-TE is provisioned by a networkcontroller that manages network elements in the transport network. Inone embodiment, nodes (e.g. along a path in the transport network)register their MAC addresses with a network controller which authorizesthe MAC addresses and records the topological location of the registerednodes. Next, when a path is to be set up, the source and destinationnodes each send a connection request to the network controller. Inresponse, the network controller decides on the path (using any pathdetermination method) and makes forwarding entries along the path in thenodes at intermediate locations, and if appropriate notifies the sourceand destination nodes.

In certain embodiments, the network controller is aware of theconfiguration of shelf 100, and a packet received on any serviceinterface of the shelf is switchable (by appropriate provisioning ofshelf 100), to any lambda on trunk interface 107Z. Similarly, a packetembedded in any lambda received at any trunk interface can be recoveredby shelf 100 and sent out on any service interface 101A-101X and108A-108P. Therefore, such lambda-aware switching of packets by shelf100, as provisioned by the network controller optimizes the use of alambda at the level of packets. Such optimization is believed to benowhere disclosed or rendered obvious by any prior art known to theinventor(s).

Inclusion of three fabrics 102, 104 and 105 all within a single shelf(which fits in a telco rack as shown in FIG. 3D) has numerous advantagesin switching traffic of a transport network, as compared to networkelements that use only two fabrics or use only one fabric within ashelf. Firstly, use of a one-fabric network element or a two-fabricnetwork element in a transport network that uses three multiplextechnologies requires traffic to be prepared for transfer between two ormore shelves. So, external interfaces are required on the two-fabric orone-fabric shelf just to externalize the traffic to be transferredtherebetween, which adds cost. In contrast, inter-shelf trafficconditioning and related external interfaces are eliminated if a networkelement that receives a signal multiplexed in three ways contains three(or more) types of fabrics in accordance with the invention. Secondly,inter-shelf configuration changes require manual labor to patch opticalfibers between two-fabric network elements and/or one-fabric networkelements. Such manual labor is eliminated when provisioning a shelf (viakeyboard) in a network that uses three multiplex technologies if theshelf internally holds three or more types of fabrics in accordance withthe invention.

Thirdly, errors can arise in communication and/or performance of manualpatching of a fiber between single-fabric or two-fabric shelves. Sucherrors are eliminated by a network controller remotely and automaticallyprovisioning a shelf that internally holds three or more types offabrics in accordance with the invention. Fourthly, single-fabricshelves of different types are typically sold by different equipmentvendors, and do not necessarily inter-operate. System integration iseliminated by use of a single shelf holding three or more types offabrics, because the fabrics exchange traffic among each other viainter-fabric circuitry also included within the same single shelf.Fifthly, presence of three or more types of fabrics in the same shelfenables optimization across three or more fabrics to a levelunattainable by optimization limited to two fabrics. For example, twoflows of packets that are respectively received at two externalinterfaces of a three-fabric shelf in accordance with the invention canbe placed on the same lambda, by appropriate provisioning. Suchoptimization of a lambda at the packet level in a transport network isbelieved to be nowhere disclosed or rendered obvious by any prior artknown to the inventors.

In one illustrative embodiment, a shelf 100 (FIG. 3B) has a midplane 211between a front region 222 and a rear region 223 of shelf 100. Rearregion 223 houses the fabrics 102, 104 and 106 while front region 222houses inter-fabric circuitries 113A and 115A and external interfacecircuitries 101A, 108A and 107A. In the illustrative embodiment shown inFIG. 3B, each of packet fabric 102, electrical fabric 104 and opticalfabric 106 is built into its own module, namely packet fabric module219, electrical fabric module 217 and optical fabric module 215. Notethat modules 215, 217 and 219 (collectively referred to as fabricmodules 214) are oriented horizontally in chassis 210, parallel to afloor thereof, and perpendicular to midplane 211.

Also, in this embodiment, the electrical-optical inter-fabric circuitry115A is built into its own module 261 that is housed in front region212. Moreover, packet-electrical inter-fabric circuitry 113A is builtinto its own module 262 that is also housed in front region 212. Modules261 and 262 (together referred to as inter-fabric modules 260) areoriented vertically relative to fabric modules 214, parallel to thesides of chassis 210, and perpendicular to midplane 211. For example, ifmidplane 211 is vertical and in the East-West direction, theninter-fabric modules 260 are also vertical but in the North-Southdirection. Fabric modules 214 are oriented horizontally andperpendicular to modules 260 as well as perpendicular to midplane 211.The just-described orientation is deliberately chosen in this embodimentto provide maximum connectivity between inter-fabric modules 260 andfabric modules 214, across midplane 211.

Note that midplane 211 of this embodiment does not have any activecircuitry to affect the optical signals and electrical signals thattravel between rear region 213 and front region 212. Midplane 211 hasbuilt in electrical connectors and optical connectors (not shown in FIG.3B) that passively couple connectors on modules in the rear region 213to corresponding connectors on modules in front region 212. In oneillustrative embodiment, midplane 211 has an upper row of opticalconnectors, and four rows of electrical connectors (all not shown inFIG. 3B) through which horizontally oriented modules 215, 217 and 219 inrear region 213 are connected to vertically oriented modules 260 and 280in front region 212. Other embodiments use other spatial arrangements ofconnectors in midplane 211, e.g. optical connectors are in a bottom rowin one embodiment while being in a middle row in another embodiment.

In embodiment shown in FIG. 3B, inter-fabric circuitry 115A iselectrically connected to a connector 221 on module 261. Connector 221in turn is optically coupled through midplane 211 to optical fabric 106on module 215. Inter-fabric circuitry 115A is also electricallyconnected to a connector 222 on module 261. Connector 222 in turn iselectrically coupled through midplane 211 to electrical fabric 104 onmodule 217. Accordingly, inter-fabric circuitry 115A can perform abridging function between optical fabric 106 and electrical fabric 104.

Also in this embodiment, inter-fabric circuitry 113A is electricallyconnected to a connector 223 on module 262. Connector 223 in turn iselectrically coupled through midplane 211 to electrical fabric 104 onmodule 217. Inter-fabric circuitry 113A is also electrically connectedto a connector 224 also on module 262. Connector 224 is electricallycoupled through midplane 211 to packet fabric 102 on module 217.Accordingly, inter-fabric circuitry 113A can perform a bridging functionbetween packet fabric 102 and electrical fabric 104.

The embodiment shown in FIG. 3B also has a number of interfaces thattransfer signals external to shelf 100, namely optical interfaces 107A,electrical interfaces 108A and packet interfaces 101A that arerespectively built into their own modules 281, 282 and 283 (collectivelyreferred to as external interface modules 280). Modules 281, 282 and 283support several different types of services, such as a packet stream inthe form of 10 Gigabit Ethernet, a TDM stream in the form of OC48 andOTU1 or STM16. Each of modules 281, 282 and 283 has an appropriate oneof connectors 225, 226 and 229 that are respectively coupled tocorresponding fabrics 106, 104 and 102 on respective modules 215, 217and 219. Modules 280 are located in front region 212 of chassis 210 ofshelf 100, oriented vertically in shelf 100, parallel to modules 260.

Note that a floor and a roof of chassis 210 has a predetermined number Nof slots in front region 212 that are flexibly used to accommodate anynumber I of inter-fabric modules 260 and any number E of externalinterface modules 280, such that N=E+I. For example, if shelf 100 is tobe used strictly as an all-optical cross-connect, all the slots are usedto hold optical external interface modules.

Shelf 100 has width and depth sufficient to fit within a telco rack 310(FIG. 3D). Telco rack 310 may be a 7 foot tall open frame consisting oftwo posts 311 and 312 separated from each other by 23 inches or 600 mm,mounted on a base 314 and connected at the top by a cross-bar 313. Rack310 is usually bolted to the floor in a central office of atelecommunications carrier. In one embodiment, shelf 100 conforms toAdvanced Telecom Computing Architecture (ATCA), a series of industryspecification standards for next generation carrier grade communicationsequipment, such as ATCA 3.0. The ATCA specification defines the physicaland electrical characteristics of such as rack and shelf form factors,power, cooling, management interfaces, and the electromechanicalspecification of ATCA-compliant boards. FIG. 3E illustrates a front viewof shelf 100 of one embodiment that holds a total of sixteen modules(such as LMX module 391) shown oriented vertically.

In another embodiment shown in FIG. 3C packet and electrical fabrics216P and 216E are both built into a single module, called hybrid fabricmodule 216. In FIG. 3C, optical fabric 106 has its own optical fabricmodule 215. Rear region 213 of the alternative embodiment has anotherhybrid fabric module 218 that also has its own packet and electricalfabrics 218P and 218E. In this embodiment, hybrid fabric module 218 isused as a hot-standby for hybrid fabric module 216, for protectionswitching in case of a failure in hybrid fabric module 216. Accordingly,modules 260 and 280 of FIG. 3C have additional connectors or additionalpins, if appropriate (relative to FIG. 3B) to connect to each of modules216 and 218. For example, module 282 has two connectors 226 and 226Rwhile module 283 also has two connectors 229 and 229R.

Electrical connectors in the midplane 211 can be, for example, MolexI-Trac connectors which eliminate the need for PCB-type traces in themidplane. Note that the same type of electrical connectors are used inconnecting through the midplane to each of two fabrics, namely packetfabric and electrical fabric. One illustrative embodiment utilizes a 6×6connector that allows for 36 differential pairs per connector. Theconnector supports serial data rates of up to 12 Gbps.

In still another embodiment (see FIG. 6B), a total of five modules arearranged horizontally in rear region 223 of shelf 100, including oneoptical fabric module and four hybrid fabric modules (as noted above inreference to FIG. 3C, each of the hybrid fabric modules has anelectrical fabric and a packet fabric). Module 281 (FIG. 3C) of someembodiments is a DWDM transport module which has an ingress portion 407and an egress portion 408 coupled to the optical fibers of an externaltrunk, labeled “Link1” in FIG. 4A. Internal to shelf 100, both portions407 and 408 are coupled to optical fabric 106 that is implemented as apassive optical mesh. Optical fabric 106 provides optical connectivitybetween all slots in shelf 100 except for slot(s) used for shelf control(as there are no optical signals transferred to/from card(s) that managethe shelf). Note that the term “module” is interchangeable with the term“line card”, e.g. optical modules 281, 281A-281C, 261 and 261A of FIG.4A are also referred to as optical “line cards.”

Ingress portion 407 of module 281 includes an optical demultiplexer 401that splits off a portion of the optical signal at a predeterminedwavelength, called “optical supervisory channel” which is used tocommunicate with other nodes in the transport network (e.g. the sourcenode for the signal on “Link1”). Next, the optical signal is amplifiedby an optical amplifier 402 in module 281, to compensate for losses onthe optical fibers of the trunk “Link1.” Thereafter, the optical signalis split by a splitter 403 in module 281, and a copy resulting fromsplitting is supplied via optical fabric 106 to each of modules281A-281C, 261 and 261A that are connected to optical fabric 106.

Note that a copy of the optical signal is also received back fromoptical fabric 106 in module 281, specifically in egress portion 408.Egress portion 408 also receives optical signals from each of modules281A-281C, 261 and 261A that are connected to optical fabric 106. Awavelength selective switch 404 in egress portion 408 is used to selectappropriate lambdas (one of each wavelength) that are together suppliedto an optical amplifier 405. Egress portion 408 also includes an opticalmultiplexer 406 that adds an optical supervisory channel and theresulting signal is then transmitted on the trunk Link1.

Accordingly, modules 281, 281A, 281B and 281C together with fabric 106form an all-optical cross-connect (also called lambda cross-connect) inshelf 100. Note that although only four all-optical modules 281, 281A,281B and 281C are illustrated, shelf 100 can accommodate up to sevenall-optical modules, with each all-optical module occupying two slots inshelf 100.

Shelf 100 is illustrated in FIG. 4A as having inter-fabric modules 261and 261A that are optically connected to optical fabric 106. Module 261contains a tunable laser 411 for conversion of an electrical signal intoan optical signal at a selected wavelength. Module 261 also includes asplitter 412 that splits the optical signal to form copies fortransmission by optical fabric 106 to each of modules 281, and 281A-281Cthat are connected to optical fabric 106. Module 261 receivesmultichannel signals from the optical fabric 106 at a tunable filter 413that selects one of the lambdas, followed by conversion into anelectrical signal by a receiver 414. Note that receiver 414 and laser411 are typically housed in a single integrated package on module 261,such as a 300-pin 10 Gbps optical transceiver.

In FIG. 4A, inter-fabric module 261 has only 4 outputs (at splitter 412)and 4 inputs (at tunable filter 413) to/from optical fabric 106 of theillustrative embodiment. Similarly, inter-fabric module 261A also hasonly 4 outputs and 4 inputs to/from optical fabric 106. The four portsin the illustrative embodiment connect each of modules 261 and 261A tomodules 281, 281A, 281B and 281C (i.e. Link1-4). Accordingly, in theillustrative embodiment, modules 261 and 261A are optically connectedonly to modules 281, and 281A-281C. Specifically, in the illustrativeembodiment modules 261 and 261A do not have optical connections amongthemselves, e.g. between their own splitters 412, 412A and tunablefilters 413 and 413A respectively. Also in the illustrative embodiment,a given each of modules 281, 281A, 281B and 281C do not have their owninternal optical connection between the respective splitter 403 andwavelength selective switch 404. However, an optical mesh 106M withinoptical fabric 106 has sufficient optical links to enable anotherembodiment having such optical connections (shown dotted in FIG. 4A),such as connection 499 within module 261A itself, and connection 498between module 261A and 261. In one example of the embodiment shown inFIG. 4A without the internal optical connections, a wavelength selectiveswitch 404 is provisionable to select 40 lambdas, from among 6 lambdasthat are received from six inter-fabric modules 261,261A . . . and fromamong 120 lambdas that are received from three all-optical modules 281A,281B and 281C.

Shelf 100 in one embodiment illustrated in FIG. 4B has four packetfabric modules 219A-219D labeled “Fabric1” . . . “Fabric4” located inthe rear region of shelf 100. Modules 219A-219D may be coupled acrossthe midplane to fourteen packet interface modules 283A-283Z labeled“Blade1” “Blade14” if present in fourteen slots available in the frontregion of shelf 100. Each of packet interface modules 283A-283Z handlespacket traffic at 20 Gbps rate, e.g. from two 10GigE links external toshelf 100. Each packet interface module 283A-283Z includes two “phy”devices that receive electrical signals from 10GigE links, a networkprocessor “NP” and a traffic manager “TM”. Each of packet interfacemodule 283A-283Z has four electrical connectors connected across themidplane to the respective packet fabric modules Fabric1 Fabric4. Eachpacket fabric module has two switch fabrics “FE” that switch packets ina non-blocking manner. Each switch fabric FE is connected to threefull-duplex ports in each packet interface module 283A-283Z. Eachfull-duplex port transmits and receives a 3.125 Gps serial signal, sothat each switch fabric FE receives a 10 Gbps signal from each packetinterface module 283A-283Z.

Note that instead of using fourteen packet interface modules, shelf 100may be used to house fourteen electrical interface modules 282A-282Z asillustrated in FIG. 4C. Accordingly, fourteen electrical interfacemodules 282A-282Z may be connected across the midplane to the fourelectrical fabric modules 217A-217D in the rear region of shelf 100. Inone embodiment, each of the four hybrid fabric modules 217A-217Dincludes an electrical crossbar labeled “X-Bar”. Each electricalcrossbar has a group of ten full-duplex ports connected to each of thefourteen electrical interface modules 282A-282Z. Each full-duplex portcan receive and transmit a 2.4 Gbps serial signal in one embodiment.Each one of modules 282A-282Z has 10 serial links to each of the fourfabric modules 282A-282Z for a total of 40 links, i.e. 20 protectedlinks per blade (i.e. per slot). In electrical fabric modules 217A-217D,each electrical crossbar transfers an electrical signal at any of itsinput ports to any of its output ports, regardless of content within thesignal. Specifically, each electrical crossbar operates at theelectrical level (analog) and not at data level (digital). The analogelectrical crossbar does not do clock recovery, data recover or CRCcheck. Instead, the electrical signal received by the electricalcrossbar is amplified at predetermined frequencies and re-shaped.Accordingly, the electrical crossbar may be used to switch an electricalsignal that carries TDM traffic, and just as easily as an electricalsignal that carries packet traffic.

In another embodiment illustrated in FIG. 4D, each of the four fabricmodules 298A-298D includes a time space switch matrix that is aware ofTDM. The TDM switch in fabric modules 298A-298D can switch anygranularity time-division-multiplexed signal, including SONET STS-1,SONET VT1.5 or OTN ODU-n. In this embodiment as well, each TDM switch ofFIG. 4D has a group of ten full-duplex ports connected to each of thefourteen TDM interface modules 299A-299Z. TDM interface modules299A-299Z, if present, are located in the fourteen slots available inthe front region of shelf 100. An example of the TDM switch isPMC-Sierra's PM5377 TSE 240 that implements a memory switch fabric withSTS-1/AU-3 switching granularity, with 96 ingress and 96 egressSTS-48/STS-12 ports.

In an illustrative embodiment shown in FIG. 4E, a fabric module 216 hasa packet fabric element 216P (implemented by two switch fabrics, notindividually shown in FIG. 4E) and also has an electrical fabricimplemented as an analog electrical crossbar 216E, as described above inreference to FIG. 3C. Each switch fabric has groups of three full-duplexports (i.e. six ports for module 216), each group connected to a trafficmanager in one of modules 297A-297Z. Each full-duplex port of the switchfabric carries a 3.125 Gbps serial signal. In addition, each crossbarhas groups of five full-duplex ports (i.e. ten ports for module 216)that are connected to an electrical switch in each of modules 297A-297Z.Each full-duplex port of the crossbar also carries a 3.125 Gbps serialsignal. Accordingly, the arrangement shown in FIG. 4E supports 20 Gbpsper slot (i.e. per traffic manager) and as there are 14 slots in theshelf, the total packet capacity is 280 Gbps. Moreover, each one ofmodules 297A-297Z has 20 protected links per blade (i.e. per slot)connected to an analog electrical crossbar.

Some embodiments of the invention use two kinds of optical interfacemodules, namely a trunk module “LXC” and a tributary module “LMX”respectively illustrated in FIGS. 5A and 5B which interface externallyto a trunk in the transport network and a tributary to client premisesrespectively, and both modules interface internally to the opticalfabric module. The trunk module “LXC” (FIG. 5A) includes an optical tap501 to split off 1% of the signal back to the faceplate so that thetelecommunication carrier can monitor it. Optically coupled to tap 501and downstream therefrom is a variable optical attenuator 502 which canbe used to appropriately change the signal strength, e.g. to avoidoverload. Attenuator 502 is typically controlled by a microcontroller(not shown) which is included in the trunk module “LXC.”

Optically coupled to attenuator 502 and downstream therefrom in theingress direction is an optical preamplifier 503, such as an Er-dopedfiber amplifier, or EDFA preamp, used to amplify a signal that has beenattenuated by transmission over a long distance. Optically coupled toamplifier 503 is a tunable dispersion compensator 507 that performsdispersion compensation of the optical signal on a per-channel basis,and the compensated signal is then returned to amplifier 503. Dependingon the embodiment, tunable dispersion compensator 507 can be fibergrating or chirped grating or Mach-Zehnder-interferometer. Examples ofcompensator 507 which may be used includes PowerShaper 3400 from Avanexand ClearSpectrum-TDC from Teraxion, Inc. While in one illustrativeembodiment the tunable dispersion compensator (“TDC”) is located withinshelf 100, an alternative embodiment transfers all signals to bedispersion compensated to an external dispersion compensating fibershelf. In the alternative embodiment, a Dispersion Compensation Module(DCM) is connected in place of the TDC, by using connectors on thefaceplate of the LXC. An example of the DCM is the Avanex PowerForm DCM.

Optically coupled to preamplifier 503 is an OSC small form factorpluggable (SFP) transceiver 504 that senses the optical supervisorychannel (OSC) at a predetermined wavelength, e.g. 1510 nm for OTN. Thepower level of the incoming OSC signal as measured by transceiver 504relative to the transmission power level of the outgoing OSC signal isused to automatically control attenuator 502, at a coarse level. A finerlevel control is performed by use of an optical channel monitor 505which monitors the power level of each wavelength (for up to 40 lambdasfor example) that are dense wave division multiplexed in the incomingoptical signal. The optical signal is then supplied to a splitter 506that splits the incoming optical signal into, for example, 9 copies, fortransmission to each of 9 fibers of the optical fabric module.

In the egress path on trunk module “LXC”, a wavelength selective switch511 (FIG. 5A) receives a number of optical signals, e.g. 1 lambda fromeach of 6 fibers and 40 lambdas from each of 3 fibers. The 3 fiberscarrying 40 lambdas originate in other trunk modules “LXC” whereas the 6fibers carrying the single lambda originate in tributary modules “LMX.”Tributary modules “LMX” are optically connected by the optical fabric ina star configuration, wherein each tributary module “LMX” is connectedto all trunk modules “LXC”. Trunk modules “LXC” are optically connectedto one another in a mesh configuration.

All signals that arrive at wavelength selective switch 511 are at anominal level of intensity. Wavelength selective switch 511 makesselections, of a number of lambdas (e.g. 40) from among all opticalsignals received from the optical fabric module. Next the selectedlambdas are amplified by an amplifier 512 and after amplification aportion of the amplified optical signal is tapped off and monitored inthe optical channel monitor 505. The intensity sensed in optical channelmonitor 505 is used to set the attenuation for each individualwavelength, in wavelength selective switch 511.

Optical channel monitor 505 (FIG. 5A) operates in the electrical domain,converting the intensity at each lambda into an electrical signal thatis sampled. Following the amplifier 512 in the egress path is anothervariable optical attenuator 513 which is currently unused but may beused if necessary. Optically coupled to attenuator 513 and downstreamtherefrom is another tap 514 that supplies 1% of the optical signal tothe faceplate for monitoring purposes. Thereafter the optical signal isdense wave division multiplexed with the optical supervisory channel(OSC) by multiplexer 515.

The OSC signal which is added by multiplexer 515 is received in thetrunk module “LXC” as an Ethernet signal, which is then supplied totransceiver 504 for conversion into an optical signal at thepredetermined wavelength, e.g. 1510 nm for OTN. The Ethernet signalwhich is transmitted/received via the OSC channel contains managementinformation communicated between shelves across the transport network.Note that up to seven trunk modules “LXC” may be used in shelf 100 ifthere are no tributary modules. Only up to four trunk modules “LXC” maybe used in shelf 100 which has one or more tributary modules “LMX”, sothat the optical fabric provides access from each trunk module to alltributaries and all trunks. Tributary modules “LMX” (discussed next)have access to all trunks but no access to other tributaries in shelf100. Hence, up to six tributary modules “LMX” may be present in shelf100.

A tributary module “LMX” (FIG. 5B) has external interfaces that supportoptical services, such as TDM (e.g. an OC48 or OTU1) or packet-based(e.g. 10GigE). An incoming optical signal to tributary module “LMX” isconverted into the electrical domain by optical transceivers in clientprocessing circuitry 521, and one or more serial signals are provided toa framer 522. Framer 522 in one embodiment conforms to G.709 and adds anOTU-2 wrapper. The OTU-2 signal is then supplied to a tunable opticaltransceiver 523 that converts this signal into the optical domain on aspecific lambda. The specific lambda that is generated by transceiver523 is provisionable by a network controller of the telecommunicationnetwork.

Next the lambda from transceiver 523 is received at a variable opticalattenuator 524 adjusts the intensity to a value appropriate for a resultof splitting by splitter 525 to be at the nominal value. Note that theoptical signal supplied by splitter 525 to the optical fabric modulecontains only one lambda, and for this reason the signal is supplied toonly those four slots in which four trunk modules “LXC” are respectivelypresent in shelf 100.

In the egress direction, an optical switch 526 receives signals fromeach of the four trunk modules “LXC” and selects one of these fouroptical signals. The selected optical signal is then transmitted to atunable optical filter 527 that selects one wavelength (e.g. from among40 lambdas) and the selected lambda is then transmitted to a transceiver523 which generates an electrical signal. The electrical signal may be aTDM signal or a packet switched signal, either of which is framed as perOTN. Hence, this electrical signal from transceiver 523 is supplied toframer 522 which strips off framing bits/bytes and supplies the signalto client side processing circuitry 521.

Client side processing circuitry 521 has a serializer-deserializer (notshown in FIG. 5B) and a laser (also not shown) to generate an opticalsignal presented at an external interface of shelf 100, e.g. as an OC48signal or an OTU1 signal. Client side processing circuitry 521 includesswitches (not shown in FIG. 5B) which can be provisioned remotely tosend any electrical signal in circuitry 521 via path 528 to hybridfabric module(s), for electrical switching and/or packet switching.Moreover, electrical signal(s) from hybrid fabric module(s) are receivedby switches in client side processing circuitry 521 and multiplexed withsignals transmitted to framer 522 and/or optical signal generated andtransmitted out of shelf 100.

In some embodiments, optical components of tributary module “LMX” (FIG.5B) are built into a daughter card, which card can be used with othermodules, such as the PMX module. Specifically, transceiver 523,attenuator 524, splitter 525, switch 526, and filter 572 are all builtinto a daughter card, while electrical components are built directlyinto the tributary module “LMX.” In one illustrative embodiment, the LMXmodule occupies only one slot, and can map four 2.5 Gbps signals into anOTU2 payload. The LMX module includes on-board tunable optics, andprovides colorless operation. In this embodiment, the LXC moduleoccupies 2 slots, and supports multiplexing 40 lambdas. The LXC modulehas automated signal amplification and dispersion compensation built in.

In some embodiments of the invention, electrical-optical inter-fabriccircuitry 115A (FIG. 3C) is implemented in an inter-fabric module “PMX”(FIG. 5C) that has interfaces to an optical fabric as well as to anelectrical fabric. Electrical-optical module “PMX” has a daughter cardsimilar or identical to that described above for module “LMX”, which iscoupled to the optical fabric module. The daughter cardreceives/transmits an OTN signal to/from an OTN mapper 532. OTN mapper532 on module “PMX” (FIG. 5C) is also similar to the above-describedframer 522 on module “LMX” (FIG. 5A). One distinction between thesemodules is that framer 522 is an OC-48/OC-192 OTU-2 framer, whereasmapper 532 is an Ethernet to OTU-2 mapper. Note that although modules“PMX” and “LMX” are similar in many respects relative to the opticalfabric module, in the illustrative embodiment they send/receivedifferent speed signals to/from the electrical fabric. Specifically, LMXsupplies 2.5 Gbps TDM signal (OC48) while PMX supplies a 10 GbpsEthernet signal.

Module “PMX” also has a multi-layer switch 531 that is coupled to OTNmapper 532 to receive/transmit an Ethernet signal embedded within theOTN signal. Multi-layer switch 531 implements an Ethernet switch locallyto/from components within the PMX module, and also performs MAC-in-MACencapsulation of Ethernet packets received by module “PMX” from anexternal interface via phy 530. After encapsulation, multi-layer switch533 switches the packets along paths that have been provisioned.Multi-layer switch 531 also receives Ethernet packets in an electricalsignal from the electrical fabric via an electrical switch 533. Thepackets from electrical switch 533 are already MAC-in-MAC encapsulatedupstream, and hence the packets from switch 533 are directly switchedthrough multi-layer switch 533 along provisioned paths. Note thatmulti-layer switch 533 of one embodiment implements the standard PBBwhile another embodiment implements the standard PBB-TE. Also,inter-fabric module “PMX” of one embodiment has optical transceivers inPHY layer circuitry 530 that are pluggably, i.e. removable so they canbe replaced with electrical transceivers. Accordingly, module PMX has an1 GbE optical interface in one embodiment and PMX has an 1 GbEelectrical interface in another embodiment.

FIG. 5C illustrates another module “PSW” that contains packet-electricalinter-fabric circuitry 113A (FIG. 3C) and has interfaces to a packetfabric as well as to an electrical fabric. Module PSW includes PHY layercircuitry 541 that includes optical transceivers, such as XFP moduleswhich form external interface(s) of the shelf, to transmit/receiveoptical signals, such as two 10 GbE signals. The optical signals areconverted into XAUI signals that are transmitted to a network processor543 either directly or through an electrical switch 543. Note that PHYlayer circuitry 541 also has pluggable transceivers which are removableso that module PSW has a 1 GbE optical interface in one embodiment andPSW has a 1 GbE electrical interface in another embodiment. Networkprocessor 543 classifies packets into flows based on their Ethernetheaders. The flows are supplied by network processor 543 to a trafficmanager 544 which performs shaping, aggregation and grooming of trafficinto one or more tunnels.

Traffic manager 544 uses 3 Gbps serial links to transfer the traffic toa packet fabric that in turn switches the traffic to one of its otherserial links. This traffic then is received by another traffic manager544 in another PSW module (although for convenience of illustration, thesame PSW module is shown in FIG. 5C). The traffic then passes throughtraffic manager 544 that performs Mac-in-Mac encapsulation and providesthe traffic (via network processor 543) to electrical switch 542 foruploading to the transport network. Note that a combination of multipletraffic managers and the packet fabric together implement an Ethernetswitch that can switch packets globally to/from modules in shelf 100 (ifthe modules have a traffic manager). Electrical switch 542 switches theelectrical signal (stream of PBB-TE frames) to the electrical fabric,which in turn transfers the signal to an electrical switch 533 in themodule “PMX.” Module “PMX” eventually transfers the packet stream to theoptical fabric module as described above.

In some embodiments, in addition to Mac-in-Mac encapsulation that isperformed in PSW and PMX modules (by network processor 543 andmulti-layer switch 531 respectively), a field programmable gate array545 is used in module PSW to create continuity check messages (CCM) foruse in management of faults in connectivity.

One illustrative embodiment uses the “CFM” standard to send CCMsperiodically to determine if a particular link is active. CCMs are usedin this embodiment to implement carrier-class recovery from faults.Switchover within 50 ms is achieved by sending CCMs at 3.3 ms intervals,with missing CCMs for 10 ms triggering alarm indication signal (AIS) andRemote Defect Indication (RDI) for fault notification, followed byswitchover which must be completed in 40 ms. An FPGA creates CCMs inhardware, faster and more predictably than can be done by the networkprocessor. The FPGA also detects CCMs and triggers a state machinewithin the network processor when a failure is detected (e.g. 3 missingCCMs).

One embodiment uses 1:1 path protection with traffic being sent only onthe working side or on the protect side, never both at the same time.This embodiment supports bidirectional switching only, wherein bridgeselectors for each direction of the protected path track each other atall time. For each path or segment it creates, the network controllersets up continuity checks for both directions. In the transport network,a segment CCM represents a B-VID. CCMs are monitored at each end of thesegment, and if CCM messages are missing for 3.5 intervals or more, thenode automatically switches to the protect path. At the same time, thenode begins to transmit RDI indication by setting the RDI bit in CCMmessages travelling in the opposite direction. Upon receiving RDI, theremote node also switches to the protect path and service is restored.As noted above, in addition to RDI, a shelf in some embodiments alsogenerates the AIS to higher layer CCM's whenever applicable. AIS can betriggered by two primary conditions: LOS at the link layer andcontinuity check failures at segment and end-to-end layers. Thisfunction is handled by the network processor in these embodiments.

Each of the modules illustrated in FIGS. 5A-5C has a host processor (notshown) that runs management software to communicate with other modulesin the shelf. Moreover, the module PSW also has an FPGA that is coupledto the traffic manager to receive and maintain statistics, e.g. on thenumber of packets being dropped per queue etc. Module PSW also has atiming core to ensure 10 Gbps interfaces in module PSW remain insynchronization with system timing in shelf 100.

In one example, traffic manager 544 in the module PSW is implemented byFAV21V available from Dune Networks. In this example, network processoris implemented by X11-d240 available from Xelerated Inc. Electricalswitch 542 is implemented by XAUI switches. Traffic manager 544 isconnected to the packet fabrics using a total of 24 full-duplex seriallinks at 3.125 Gbps in groups of 6, with one group being connected toeach packet fabric. In addition, electrical switch 542 in module PSW isconnected to the electrical fabrics using a total of 16 full-duplexserial links at 3.125 Gbps in groups of 4, with one group beingconnected to each electrical fabric. Moreover, in this example,multi-layer switch 531 is implemented by BCM56514 available fromBroadcom Corporation. OTN framer 532 of this example is implemented byPEMAQUID available from AMCC. Electrical switch 533 is implemented bytwo M21453 (one each for egress and ingress) available from MindspeedTechnologies, Inc. which implements a 12×12 fully non-blockingelectrical crossbar. G.709 Framer (or OTN Mapper) 522 of this example isimplemented by the Cortina Tenabo.

In the example, packet fabric 102 is implemented by two switch fabricsFE200 available from Dune Networks. Also in the example, electricalfabric 104 is implemented by VSC3172 available from VitesseSemiconductor Corporation. This chip is a 72×72 fully non-blockingelectrical crossbar, and only 70 links are used at 10 Gbps speed, with 5links per slot (as there are 14 slots in the front of the chassis ofshelf 100, to hold interface modules and inter-fabric modules). Of the 5links, 4 links may be used to switch XAUI signals from/to the PMX moduleor PSW module. In each slot, the fifth link from the crossbar in anillustrative embodiment is directly connected to an external 10 GbpsEthernet optical interface on each of the PMX module and PSW module,whichever is present in the slot.

In one example, the following parts are used to implement the LMX andLSW illustrated in FIGS. 5A and 5B, WSS: Optium DWP100, Preamp EDFA:Avanex PureGain 2600, Booster EDFA: Avanex PureGain 1500, OpticalChannel Monitor: Aegis CTM-4050, Variable Optical Attenuator: JDSU MATTVOA, Tunable Filter: JDSU VCF100, Transceiver: Optium 300-pin 10G NRZTransponder. Any devices not explicitly described herein are commodityitems available from several vendors.

In the illustrative embodiment, a shelf 100 is illustrated in FIG. 6Awhich shows a side view in the direction “B-B” shown by arrows in FIG.3E. FIG. 6B shows a rear view of shelf 100, with onehorizontally-mounted optical fabric module 601 and fourhorizontally-mounted hybrid modules 602. A number of rear terminalmodules 603 are mounted vertically, and these modules are typicallyconnected to corresponding modules in the front region of shelf 100. Forexample, the module PMX has a multi-layer switch 531 which can beprovisioned to drive copper interfaces (e.g. 1 GbE) in rear terminalmodules 603.

FIGS. 6C and 6D illustrate a front view and a perspective of a frontregion of shelf 100. As shown in FIG. 6C, the midplane in shelf 100 hasa row 603 of optical connectors and four rows of electrical connectors604. In FIGS. 6C and 6D, one module LMX is shown installed. FIGS. 6E and6F illustrate a rear view and a perspective of a rear region of theshelf, with three rows 605 of electrical connectors visible, one hybridmodule and the optical fabric module installed. FIG. 6G is same as FIG.6C with the addition of section line A-A along which is shown across-sectional view in FIG. 6H. FIG. 6G also shows the three fan trays606A-606C located at the bottom of the shelf, providing air flow fromthe front bottom to the back top of the shelf. The fans in trays606A-606C operate under software control, and the fans speed up in theevent of a failure. Note that three such shelves can be stacked in a 7′rack, with room left in the rack for fuse and alarm panel and AC outletassembly. FIG. 6I is an exploded view of the shelf of FIGS. 6A-6G,wherein top and side portions of the shelf are removed to improveclarity.

In summary, the benefits of a single shelf 100 uses optical, electricaland packet fabrics all three types in a single chassis as shown in FIGS.6A-6I are several. The shelf supports DWDM trunk interfaces which canscale from one to seven by simply adding additional LXC modules to thechassis. Wavelengths from any trunk can be switched to any other trunkacross the optical backplane. Client signals can be added to the DWDMtrunks through the use of various transponder modules LMX. Thetransponder modules LMX are accommodated in the same shelf as trunkmodules LXC (up to degree-4). In addition to the optical functions, theshelf supports a flexible data plane in the same chassis. Four hybridmodules can be installed into the same shelf in the rear, to supporthigh bandwidth applications with 3+1 protection. Alternatively, fabricmodules for multiple types of data (Packet, TDM) can be installed with1:1 protection in the rear. In the absence of an active fabric element,the slots can be used to implement a passive interconnect.

One example of shelf 100 is implemented in based on the followingindustry standards, each of which is attached hereto, and isincorporated by reference herein in its entirety: ITU-T RecommendationG.709/Y.1331—Interfaces for the Optical Transport Network (OTN), March2003; ITU-T Recommendation G.975—Forward Error Correction for SubmarineSystems, October 2000; ITU-T Recommendation G.975.1—Forward ErrorCorrection for high bit-rate DWDM submarine systems, February 2004; IEEE802.1 ah/D4.0—DRAFT Amendment to IEEE Std 802.Q—REV, Virtual BridgedLocal Area Networks—Amendment 6: Provider Backbone Bridges, Nov. 21,2007; and IEEE 802.1 Qay/D2.0—DRAFT Amendment to IEEE Std 802.Q—2005,Virtual Bridged Local Area Networks—Amendment: Provider Backbone BridgeTraffic Engineering, Feb. 15, 2008. Implementation of such industrystandards in a single shelf that uses optical, electrical and packetfabrics all three types in a single chassis is believed to be nowheredisclosed or rendered obvious in any prior art known to the inventor(s).

Numerous modifications and adaptations of the embodiments describedherein will become apparent to the skilled artisan in view of thisdisclosure. For example, some embodiments support various legacyservices such as Frame Relay and ATM by using Pseudowire (“PW”) directlyover PBB-TE. Also, although some embodiments use PBB-TE over G.709 overDWDM, other embodiments use Transport MPLS (“TMPLS”) over G.709 overDWDM. Moreover, although G.709 is used to frame the signal, any otherwrapper may be used in other embodiments. Also, the term “packet”, asused in the description above of many illustrative embodiments, refersto units of data having MAC header(s), for example, an IP packet whenencapsulated in an Ethernet frame. However, depending on the embodiment,units of data with other types of headers may also be switched in aconnection oriented manner as described herein by a packet switch, e.g.some embodiments of packet switches used in a shelf 100 switch dataunits of fixed size using virtual circuits, similar to ATM but withlarger size, such as 1500 bytes for a MAC frame conforming to Ethernet.Numerous modifications and adaptations of the embodiments describedherein are encompassed by the scope of the invention.

1-20. (canceled)
 21. A system adapted to provision traffic in atransport network through at least three switching modes in a transportnetwork, the system comprising: a network controller communicativelycoupled to a plurality of network elements in the transport network,wherein each of the plurality of network elements comprise a pluralityof an optical fabric adapted to perform lambda switching of the traffic,an electrical fabric adapted to perform time-division-multiplexed (TDM)of the traffic, and a packet fabric adapted to perform packet switchingof the traffic; wherein the network controller is adapted to provisionthe traffic through the transport network via the optical fabric, theelectrical fabric, and the packet fabric at each of the plurality ofnetwork elements.
 22. The system of claim 21, wherein the networkcontroller is adapted to provision each network element on a path for aflow of encapsulated packets, based on one or more fields in a MediaAccess Control header.
 23. The system of claim 21, wherein the networkcontroller is adapted to provision each network element in the transportnetwork a specific path that is identified in Provider BackboneBridging-Traffic Engineering.
 24. The system of claim 21, wherein thenetwork controller is aware of a configuration of each of the pluralityof network elements.
 25. The system of claim 21, wherein the networkcontroller is adapted to provision each lambda at each of the pluralityof network elements at a packet level.
 26. The system of claim 21,wherein the network controller is adapted to automatically provision anetwork element remotely based on fiber patching on the network element.27. The system of claim 21, wherein the network controller is adapted toautomatically provision a lambda for each transceiver at the pluralityof network elements.
 28. The system of claim 21, wherein the networkcontroller is adapted to provision continuity checks for 1:1 pathprotection on the traffic in the transport network.
 29. The system ofclaim 21, wherein the network controller is adapted to provision twoflows of packets that are respectively received at two externalinterfaces of a three-fabric shelf on a same lambda.
 30. A transportnetwork supporting traffic through at least three switching modes, thetransport network comprising: a plurality of network elements, whereineach of the plurality of network elements comprise a plurality of anoptical fabric adapted to perform lambda switching of the traffic, anelectrical fabric adapted to perform time-division-multiplexed (TDM) ofthe traffic, and a packet fabric adapted to perform packet switching ofthe traffic; and a network controller communicatively coupled to theplurality of network elements; wherein the network controller is adaptedto provision the traffic through the transport network via the opticalfabric, the electrical fabric, and the packet fabric at each of theplurality of network elements.
 31. The transport network of claim 30,wherein the network controller is adapted to provision each networkelement on a path for a flow of encapsulated packets, based on one ormore fields in a Media Access Control header.
 32. The transport networkof claim 30, wherein the network controller is adapted to provision eachnetwork element in the transport network a specific path that isidentified in Provider Backbone Bridging-Traffic Engineering.
 33. Thetransport network of claim 30, wherein the network controller is awareof a configuration of each of the plurality of network elements.
 34. Thetransport network of claim 30, wherein the network controller is adaptedto provision each lambda at each of the plurality of network elements ata packet level.
 35. The transport network of claim 30, wherein thenetwork controller is adapted to automatically provision a networkelement remotely based on fiber patching on the network element.
 36. Thetransport network of claim 30, wherein the network controller is adaptedto automatically provision a lambda for each transceiver at theplurality of network elements.
 37. The transport network of claim 30,wherein the network controller is adapted to provision continuity checksfor 1:1 path protection on the traffic in the transport network.
 38. Thetransport network of claim 30, wherein the network controller is adaptedto provision two flows of packets that are respectively received at twoexternal interfaces of a three-fabric shelf on a same lambda.
 39. Amethod adapted to provision traffic in a transport network through atleast three switching modes in a transport network, the methodcomprising: providing a network controller communicatively coupled to aplurality of network elements in the transport network, wherein each ofthe plurality of network elements comprise a plurality of an opticalfabric adapted to perform lambda switching of the traffic, an electricalfabric adapted to perform time-division-multiplexed (TDM) of thetraffic, and a packet fabric adapted to perform packet switching of thetraffic; wherein the network controller is adapted to provision thetraffic through the transport network via the optical fabric, theelectrical fabric, and the packet fabric at each of the plurality ofnetwork elements.
 40. The method of claim 39, wherein the networkcontroller is adapted to provision each lambda at each of the pluralityof network elements at a packet level.